Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci(e2e): Migrate tests to Jest #53702

Closed
wants to merge 31 commits into from
Closed

ci(e2e): Migrate tests to Jest #53702

wants to merge 31 commits into from

Conversation

scinos
Copy link
Contributor

@scinos scinos commented Jun 15, 2021 •
edited
Loading

Changes proposed in this Pull Request

Migrate Calypso e2e tests to Jest.

Main differences

Jest and Mocha API are very similar. There are a few differences though:

  • Jest don't have this inside the test. We can't store values there (eg: no more this.driver = ...).
  • Hooks don't have names.
  • before/after hooks are named beforeAll/afterAll
  • Use jest.setTimeout() instead of this.timeout() to set test timeouts
  • For hooks, the timeout is passed as the second parameter

Pros:

  • Seems to be a bit faster. I got a build in 3m:10s, while trunk is about 3m:30s. This doesn't even use Jest caching, so it may end up running in less than 3 minutes.

  • jest-teamcity reporter adds the test filename as part of the test name in TeamCity. This can help identify which tests is failing:
    image

  • Log are much easier to read, even when running multiple tests in parallel. It is easy to see what was the test suite, what step failed, the error and the stack trace:
    image

Cons:

  • jest-teamcity reporter adds the test filename as part of the test name... which means that renaming a file will erase the history of the test in TeamCity, as it will appear as a new test.

  • Test that are in specs/specs-calypso but don't have the @parallel tag will appear as disabled in TC UI.

@matticbot
Copy link
Contributor

This PR modifies the release build for wpcom-block-editor

To test your changes on WordPress.com, run install-plugin.sh wpcom-block-editor e2e/migrate-to-jest on your sandbox.

To deploy your changes after merging, see the documentation: PCYsg-l4k-p2

@matticbot
Copy link
Contributor

matticbot commented Jun 15, 2021 •
edited
Loading

Link to Calypso live: https://calypso.live?image=registry.a8c.com/calypso/app:build-9541
Link to Jetpack Cloud live: https://calypso.live?image=registry.a8c.com/calypso/app:build-9541&env=jetpack

scinos
scinos commented Jun 15, 2021
View reviewed changes
test/e2e/.eslintrc.js
{
plugins: [ 'mocha' ],
files: [
'test/e2e/specs/specs-jetpack/*',
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Enable mocha and disable jest plugin & rules for these paths

@scinos scinos changed the base branch from trunk to ci/e2e/refactor-mocha-hooks June 15, 2021 13:54
@scinos scinos mentioned this pull request Jun 15, 2021
Closed
@matticbot
Copy link
Contributor

This PR does not affect the size of JS and CSS bundles shipped to the user's browser.

Generated by performance advisor bot at iscalypsofastyet.com.

@scinos scinos force-pushed the e2e/migrate-to-jest branch 2 times, most recently from 85ed021 to 1ace87c Compare June 16, 2021 04:49
Base automatically changed from ci/e2e/refactor-mocha-hooks to trunk June 16, 2021 05:46
@scinos scinos force-pushed the e2e/migrate-to-jest branch from 1ace87c to 73128bb Compare June 16, 2021 05:48
@scinos scinos marked this pull request as ready for review June 16, 2021 10:40
@scinos scinos requested review from WunderBart and a team as code owners June 16, 2021 10:40
@scinos scinos requested a review from a team June 16, 2021 10:40
@matticbot matticbot added [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. labels Jun 16, 2021
scinos
scinos commented Jun 16, 2021
View reviewed changes
test/e2e/lib/hooks/video/video-recorder.js Outdated
@@ -19,7 +19,6 @@ export const buildHooks = ( displayNum ) => {
let ffVideo;

const startVideoRecording = async () => {
console.log( `Start video recording on port :${ displayNum }}` );
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very noisy

scinos
scinos commented Jun 16, 2021
View reviewed changes
test/e2e/lib/jest/environment-fail-fast.js Outdated
@@ -0,0 +1,17 @@
const JestEnvironmentNode = require( 'jest-environment-node' );
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is used to implement failfast (abort current spec if it fails, don't touch other specs). Inspired by jestjs/jest#6527 (comment)

@scinos scinos force-pushed the e2e/migrate-to-jest branch from 9eb9116 to 28646d2 Compare June 16, 2021 11:46
@matticbot
Copy link
Contributor

This PR modifies the release build for editing-toolkit

To test your changes on WordPress.com, run install-plugin.sh editing-toolkit e2e/migrate-to-jest on your sandbox.

To deploy your changes after merging, see the documentation: PCYsg-mMA-p2

@sarayourfriend
Copy link
Contributor

The desktop tests failed, is that expected?

@scinos scinos marked this pull request as draft June 21, 2021 03:52
@scinos scinos removed the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Jun 21, 2021
@scinos scinos force-pushed the e2e/migrate-to-jest branch from 11452d6 to b75b7d6 Compare June 23, 2021 12:38
@matticbot matticbot added the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Jun 23, 2021
@scinos scinos force-pushed the e2e/migrate-to-jest branch 2 times, most recently from dff57c5 to 99c3353 Compare June 25, 2021 04:29
@scinos scinos force-pushed the e2e/migrate-to-jest branch from 00fbbe4 to a5b4965 Compare July 5, 2021 05:08
@scinos scinos self-assigned this Jul 5, 2021
@scinos
Copy link
Contributor Author

scinos commented Jul 26, 2021

Abandoned, we migrated Playwright tests instead.

@scinos scinos closed this Jul 26, 2021
@sirreal sirreal deleted the e2e/migrate-to-jest branch January 14, 2025 17:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@WunderBart WunderBart Awaiting requested review from WunderBart

Assignees

@scinos scinos

Labels
[Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@scinos @matticbot @sarayourfriend